Algebraic Techniques for E cient Inference in Bayesian Networks

A number of exact algorithms have been developed to perform probabilistic inference in Bayesian belief networks in recent years. These algorithms use graph-theoretic techniques to analyze and exploit network topology. In this paper, we examine the problem of e cient probabilistic inference in a belief network as a combinatorial optimization problem, that of nding an optimal factoring given an algebraic expression over a set of probability distributions. We de ne a combinatorial optimization problem, the optimal factoring problem, and discuss application of this problem in belief networks. We show that optimal factoring provides insight into the key elements of e cient probabilistic inference, and present simple, easily implemented algorithms with excellent performance. We also show how use of an algebraic perspective permits signi cant extension to the belief net representation. 1 Probabilistic inference in belief networks The problem of e cient inference in belief networks is usually formulated graph-theoretically as a problem of exploiting typically sparse network topology. In this paper we argue that the problem can be pro tably explored algebraically, and present some results we have developed in the course of such exploration. A belief network is a directed acyclic graph containing a set of nodes, a set of arcs and a set of numeric probability distributions. A node represents a domain variable with mutually exclusive and exhaustive values. Arcs and numeric probability distributions describe the probabilistic relationship among nodes. A belief network is singly-connected if there is at most one undirected path between any two nodes, otherwise it is called multiply-connected. Figure 1 shows a simple multiplyconnected belief network. Probabilistic inference in a belief network is the task of computing a marginal or conditional probability distribution across some subset of the variables in the network, given evidence on some subset of the remaining variables. A belief network is a compact representation of a full joint probability distributor over the n domain variables in the belief network. In particular, the full joint probability distribution can be calSeveral key results, including the formalization of the problem, were developed in collaboration with former Phd student Zhaoyu Li Z Z Z Z~ = Z Z Z Z~ Z Z ZZ~ =